Predicting the Performance of Minimax and Product in Game-Tree Searching'

The discovery t ha t the minimax decision rule performs poorly in some games has sparked interest in possible al ternatives to minimax. Until recently, the only games in which minimax was known to perform poorly were games which were mainly of theoretical interest. However, this paper reports results showing poor performance of minimax in a more common game called kalah. For the kalah games tested, a non-minimax decision rule called the product rule performs significantly bet ter than minimax. This paper also discusses a possible way to predict whether or not minimax will perform well in a game when compared to product . A parameter called the rate of heuristic flaw (rhf) lias been found to correlate positively with the performance of product against minimax. Both analytical and experimental results are given tha t appear to support the predictive power of rhf. 1 . I n t r o d u c t i o n Since the discovery of pathological games [3/1], two questions have a t t racted a fair amount of research interest. First, is it beneficial to search deeper in various real games? Second, since game tree pathology occurs when using the well known minimax back-up rule in some games, are there al ternatives that might do better? Pearl [7,8] suggested tha t one should consider the product rule as a way to combine values from an evaluation function. Nau, Purdom, and Tzeng [5] did some experiments and found tha t in a class of board spli t t ing games, product almost always performed bet ter than minimax and tha t the product rule avoided pathology. But so far, poor performance of minimax relative to other decision rules has not been observed in games people actually play. By the results of experiments on a more common game called kalah, Slagle and Dixon [10] found tha t a decision procedure called "M & N " performed significantly better than minimax. However, the M & N rule has a great resemblance to minimax. Underlying the above questions is a more fundamental issue: Why does minimax perform well in many games, and why does it perform poorly in some others? Avoidance of sibling dependencies [6] and avoidance of traps (8) have been proposed to be causes of peculiarities such as game tree pathology and the bet ter performance of non-minimax back-up rules in some games. But these two characteristics have more to do with the structure of the game tree itself than they have to do with the heuristics being used. Abramson 's studies [l] on board splitting games showed tha t one can avoid pathology by improving the heuristics. Apparent ly this improvement of heuristics can be credited to the existence and detection of " t r a p s . " 1 This work was supported in part by an NFS Presidential Young Investigator Award to Dana S. Nau, with matching funds provided by IBM, General Motors, and Mart in Mariet ta; and by NSF grant NSFD CDR-85-00108. This paper explores the issues of trap avoidance and sibling dependencies on the game of kalah and two modifications of kalah. By means of Monte Carlo studies on these games, three interesting results have been observed: (1) On the average, the product rule performs better than minimax on the kalah games tested. (2) For the games tested, a parameter called the rate of heuristic flaw appears to be a good predictor of whether minimax will do better than the product rule. (3) For the games tested, the existence of traps correlates negatively with the performance of minimax playing against product a result that appears to conflict with what one might predict from Pearl's and Abramson's studies. 2. A parameter for game tree measurement 2.1. The rate of heuristic flaw Let G be a zero sum, perfect information game between two players called max and min, and assume that G has no ties. Let e(.) be a static evaluation function for G. If n is a board position in G, let Win(n) and Loss(n) denote the events that n is a forced win for max or forced loss for max respectively. Consider two board positions, m and n. If